{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Entropy maximization\n",
    "\n",
    "A derivative work by Judson Wilson, 6/2/2014.<br>\n",
    "Adapted from the CVX example of the same name, by Joëlle Skaf, 4/24/2008.\n",
    "\n",
    "## Introduction\n",
    "\n",
    "Consider the linear inequality constrained entropy maximization problem:\n",
    "    \\begin{array}{ll}\n",
    "    \\mbox{maximize}   & -\\sum_{i=1}^n x_i \\log(x_i) \\\\\n",
    "    \\mbox{subject to} & \\sum_{i=1}^n x_i = 1 \\\\\n",
    "                      & Fx \\succeq g,\n",
    "    \\end{array}\n",
    "where the variable is $x \\in \\mathbf{{\\mbox{R}}}^{n}$.   \n",
    "\n",
    "This problem can be formulated in CVXPY using the `entr` atom.\n",
    "\n",
    "## Generate problem data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "import cvxpy as cp\n",
    "import numpy as np\n",
    "\n",
    "# Make random input repeatable. \n",
    "np.random.seed(0) \n",
    "\n",
    "# Matrix size parameters.\n",
    "n = 20\n",
    "m = 10\n",
    "p = 5\n",
    "\n",
    "# Generate random problem data.\n",
    "tmp = np.random.rand(n)\n",
    "A = np.random.randn(m, n)\n",
    "b = A.dot(tmp)\n",
    "F = np.random.randn(p, n)\n",
    "g = F.dot(tmp) + np.random.rand(p)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Formulate and solve problem"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "     pcost       dcost       gap    pres   dres\n",
      " 0:  0.0000e+00 -2.8736e+00  2e+01  1e+00  1e+00\n",
      " 1: -6.0720e+00 -5.9687e+00  2e+00  8e-02  2e-01\n",
      " 2: -5.4688e+00 -5.5883e+00  4e-01  8e-03  4e-02\n",
      " 3: -5.4595e+00 -5.4889e+00  5e-02  6e-04  1e-02\n",
      " 4: -5.4763e+00 -5.4816e+00  1e-02  1e-04  5e-03\n",
      " 5: -5.4804e+00 -5.4809e+00  1e-03  1e-05  2e-03\n",
      " 6: -5.4809e+00 -5.4809e+00  3e-05  5e-07  3e-04\n",
      " 7: -5.4809e+00 -5.4809e+00  4e-07  6e-09  1e-05\n",
      " 8: -5.4809e+00 -5.4809e+00  4e-09  6e-11  3e-07\n",
      " 9: -5.4809e+00 -5.4809e+00  4e-11  6e-13  4e-09\n",
      "Optimal solution found.\n",
      "\n",
      "The optimal value is: 5.480901486350394\n",
      "\n",
      "The optimal solution is:\n",
      "[0.43483319 0.66111715 0.49201039 0.36030618 0.38416629 0.30283658\n",
      " 0.41730232 0.79107794 0.76667302 0.38292365 1.2479328  0.50416987\n",
      " 0.68053832 0.67163958 0.13877259 0.5248668  0.08418897 0.56927148\n",
      " 0.50000248 0.78291311]\n"
     ]
    }
   ],
   "source": [
    "# Entropy maximization.\n",
    "x = cp.Variable(shape=n)\n",
    "obj = cp.Maximize(cp.sum(cp.entr(x)))\n",
    "constraints = [A*x == b,\n",
    "               F*x <= g ]\n",
    "prob = cp.Problem(obj, constraints)\n",
    "prob.solve(solver=cp.CVXOPT, verbose=True)\n",
    "\n",
    "# Print result.\n",
    "print(\"\\nThe optimal value is:\", prob.value)\n",
    "print('\\nThe optimal solution is:')\n",
    "print(x.value)"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.6"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 1
}
